The Appendix is organized as follows.

It offers a data-independent measure of the complexity of the learned latent variable description, giving the log of the effective description length.

To achieve learning in deeper networks we have used a curriculum on random and MNIST data. Next, we use a deep network and provide intermediate errors by a ground truth network. Finally, we remove intermediate errors and use the RNN's intermediate predictions that are now close to the ground truth. Figure 12 provides the entire meta test training trajectories for a subset of all configurations. Furthermore, in Figure 13 we show the cumulative accuracy on the first 100 examples.

Neural Information Processing Systems http://nips.cc/

RRTop-k is " -DP as desired. The training set contains 60,000 examples and the test set contains 10,000. On MNIST, Fashion MNIST, and KMNIST, we train the models with mini-batch SGD with batch size 265 and momentum 0.9. On CIFAR-10, we use batch size 512 and momentum 0.9, and train for 200 epochs. The learning rate is scheduled according to the widely used piecewise constant with linear rampup scheme.

In Table 8, we include results of a set of simple experiments for V AE trained on SVHN.

The number of cleaned counter-examples across data sets and models is more than 30% of the total number of cleaned examples. FIM-based approaches outperform the LISSA estimator. FIM, which is difficult to store and invert. Figure 3 shows the results of the evaluation of Top Fisher, Practical Fisher and nearest neighbor (NN). As reported in the main text, Practical Fisher lags behind Top Fisher in all cases.